Learning agents for the multi-mode project scheduling problem

The integration of machine learning methods into meta-heuristic search techniques is currently an important topic in the field of combinatorial optimisation. It is often referred to as intelligent optimisation [1]. Learning has the potential of helping the search process in several ways. Searching for instance, can become more generic when online learning and automation mechanisms tune the critical parameters of the search process themselves and avoid expensive manual finetuning. Some techniques also have the ability of learning to explain why, when and how the search is effective, resulting in better documented search. In this paper [6] we integrate multi-agent reinforcement learning and local search for building schedules for the multi-mode resource-constrained project scheduling problem (MRCPSP). In the last few decades, the resource constrained project scheduling problem (RCPSP) has become an attractive subject in operational research. It considers scheduling a project’s activities while respecting the resource requirements and the precedence relations between the activities. The academic problem has many relevant real world counterparts in the building and consultancy sector, for example, which results in a challenging list of instances with varying optimisation objectives. The MRCPSP is a generalized version of the RCPSP, where each activity can be performed in one out of a set of modes, with a specific activity duration and resource requirements (e.g. 2 people each with a shovel need 6 days to dig a pit, while 4 people each with a shovel and one additional wheelbarrow need only 2 days). The RCPSP is shown to be an NP-hard optimisation problem [2], thus so is the MRCPSP. Demeulemeester and Herrroelen [3] present a comprehensive research handbook on project scheduling. The idea we propose here is to learn a good constructive heuristic that can be further refined by local search techniques for large scale scheduling problems. With the latter goal in mind, a network of distributed reinforcement learning agents was set up, in which agents cooperate to jointly learn a well performing heuristic. As we will show, our method generates results that are comparable to those obtained by the best-performing finetuned algorithms found in the literature, which confirms the above mentioned positive presumptions of using machine learning in search.